Projection of a plurality of structured light patterns

ABSTRACT

Systems and methods may provide for projection of a plurality of structured light patterns. In one example, the method may include generating a low-resolution pattern image utilizing a returned image, wherein the low-resolution pattern image is an approximation of an image that would have been generated utilizing a low-resolution pattern and generating a high-resolution pattern image utilizing a preprocessed returned image and a preprocessed low-resolution pattern image, wherein high-resolution pattern image is an approximation of an image that would have been generated utilizing a high-resolution pattern.

CROSS REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 13/997,872 filed Jun. 25, 2013, which is the National Stage ofInternational Application No. PCT/US2012/031007, filed Mar. 28, 2012.

BACKGROUND

Technical Field

Embodiments generally relate to structured light patterns. Moreparticularly, embodiments relate to providing projection of a pluralityof structured light patterns.

Discussion

Structured light may be used to project a known pattern of pixels onto atarget image. When an image is returned, the manner in which the patternof pixels is deformed may allow a vision system to calculate depth andsurface information in the target image.

However, in some cases, particular aspects of a scene may require moreparticular consideration. For example, in order to appear in an equalresolution as nearer objects, objects within the scene that are fartheraway may require finer features. In these cases, utilizing one patternmay be inadequate.

BRIEF DESCRIPTION OF THE DRAWINGS

The various advantages of the embodiments of the present invention willbecome apparent to one skilled in the art by reading the followingspecification and appended claims, and by referencing the followingdrawings, in which:

FIG. 1 is a block diagram of an example of a computer system that mayprovide projection of a plurality of structured light patterns inaccordance with an embodiment of the invention;

FIG. 2 is a illustration of an example of a high-resolution pattern inaccordance with an embodiment of the invention;

FIG. 3 is a illustration of an example of a low-resolution pattern inaccordance with an embodiment of the invention;

FIG. 4 is a illustration of an example of a combination pattern inaccordance with an embodiment of the invention; and

FIG. 5 is a flowchart of an example of a method of that may provideprojection of a plurality of structured light patterns in accordancewith an embodiment of the invention.

DETAILED DESCRIPTION

Turning now to FIG. 1, a block diagram of an example of a computersystem 100 that may provide for projection of a plurality of structuredlight patterns is shown. The computer system 100 may include a computingdevice 10 and a fabrication device 30.

The computing device 10 may be, among other things, any programmablemachine that may carry out a sequence of logical operations. Examples ofthe computing device 10 may include a laptop, desktop, personal digitalassistant (PDA), media player, imaging device, mobile Internet device(MID), any smart device such as a smart phone, smart tablet, smart TV,or the like. In this embodiment, the computing device may be a desktopcomputer. The computing device 10 may include a processing component 11,an interface 12, a sensor 13, a camera 14, a projector 15, a low-passfilter component 16, and a memory component 17.

The processing component 11 may include at least one computer processorthat may be utilized to execute computer-readable executableinstructions. For example, as will be discussed in greater detail, theprocessing component 11 may be utilized to execute an applicationdirected to projection of a plurality of structured light patterns. Theinterface 12 may allow a user to interact with the computing device 10.In this embodiment, the interface 12 may be a computer monitordisplaying a graphical user interface (GUI). The sensor 13 may be asensor to detect transmission of electromagnetic energy. In thisembodiment, the sensor 13 may be a photo detector.

The camera 14 may be any device configured to capture an image or avideo feed of an object or image. The projector 15 may be an opticaldevice that projects an image onto a surface. In this embodiment, theprojector 15 may include a projection screen.

The low-pass filter component 16 may be an electronic filter that passesthrough low-frequency signals, but attenuates (i.e., reduces amplitudeof) signals with frequencies higher than a cutoff frequency. In thisembodiment, the low-pass filter component may an electronic circuit.

The fabrication device 30 may be a device configured to fabricate astructured light pattern mask. In this embodiment the fabrication device30 may be utilized to fabricate a combination mask 40.

The computing device memory 17 may be any device configured to storedata. In this case, the first computing device memory 17 may store,among other things, a projection application 18, a first pattern 19, asecond pattern 20, and a combination pattern bitmap 21. The computingdevice memory 17 may also store a returned image 22, a second patternimage 23, a preprocessing application 24, a preprocessed returned image25, a preprocessed second pattern image 26, and a first pattern image27.

The projection application 18 may be a software application that may beexecuted by a processor (e.g., the processing component 11). As will bediscussed in greater detail below, the projection application 18 may beconfigured to generate the combination pattern bitmap 21, and utilizethe combination pattern bitmap 21 to create the combination mask 40. Inaddition, the projection application 18 may also be configured utilizethe combination pattern bitmap 21 and the combination mask 40 togenerate the second pattern image 23 and the first pattern image 27.

The first pattern 19 may be a pattern that is used to create thecombination pattern bitmap 21 and the combination mask 40. In thisembodiment, the first pattern 19 may be a high-resolution pattern, andmay take the form of a bitmap image. FIG. 2 illustrates an example of ahigh-resolution pattern 200.

The second pattern 20 may be a second pattern that may be used to createthe combination pattern bitmap 21 and the combination mask 40. In thisembodiment, the second pattern 20 may be a low-resolution pattern, andmay take the form of a bitmap image. FIG. 3 illustrates an example of alow-resolution pattern 300.

The combination pattern bitmap 21 may be a bitmap that may be generatedusing the first pattern 19 and the second pattern 20, and may be used tofabricate the combination mask 40 (e.g., using the fabrication device30). The combination pattern bitmap 21 may be generated by theprojection application 18. FIG. 4 illustrates an example of acombination pattern 400.

Turning back to FIG. 1, the returned image 22 may be an image that isreturned as a result of a projection (e.g., using the projector 15).More particularly, in this embodiment, the returned image 22 may be animage that is returned using the combination mask 40.

The second pattern image 23 may be an image that may be generated by theprojection application 18. More particularly, the second pattern image23 may be the result of application the low-pass filter component 16 tothe returned image 22, and may represent an approximation of an imagethat would have been generated by the second pattern 20 alone.

The preprocessing application 24 may be an application utilized toprepare an image for processing. For example, in this embodiment, thepreprocessing application 24 may be utilized to, among other things,perform a thresholding operation to convert a non-binary image to abinary (e.g., bitmap) format.

The preprocessed returned image 25 may be an image representation of thereturned image 22 after having been passed through the preprocessingapplication 24. Similarly, the preprocessed second pattern image 26 maybe an image representation of the second pattern image 23 after havingbeen passed through the preprocessing application 24.

The first pattern image 27 may be an image that may be generated by theprojection application 18, for example, using the preprocessed returnedimage 25 and the preprocessed second pattern image 26. The first patternimage 27 may represent an approximation of an image that would have beengenerated by the first pattern 19 alone.

The arrangement and numbering of blocks depicted in FIG. 1 is notintended to imply an order of operations to the exclusion of otherpossibilities. Those of skill in the art will appreciate that theforegoing systems and methods are susceptible of various modificationsand alterations.

For example, in the embodiment illustrated in FIG. 1, the low-passfilter component may be an electronic circuit. However, this need notnecessarily be the case. In other embodiments, the low-pass filtercomponent may be a digital filter.

Turning now to FIG. 5, a flowchart of an example of a method that mayprovide projection of a plurality of structured light patterns inaccordance with an embodiment of the invention is shown. The methodmight be implemented as a set of logic instructions stored in a machine-or computer-readable storage medium such as random access memory (RAM),read only memory (ROM), programmable ROM (PROM), firmware, flash memory,etc., in configurable logic such as programmable logic arrays (PLAs),field programmable gate arrays (FPGAs), complex programmable logicdevices (CPLDs), in fixed-functionality logic hardware using circuittechnology such as application specific integrated circuit (ASIC),complementary metal oxide semiconductor (CMOS) or transistor-transistorlogic (TTL) technology, or any combination thereof. For example,computer program code to carry out operations shown in the method may bewritten in any combination of one or more programming languages,including an object oriented programming language such as Java,Smalltalk, C++ or the like and conventional procedural programminglanguages, such as the “C” programming language or similar programminglanguages.

The process may begin at processing block 2000. At processing block2010, a projection application, such as the projection application 18(FIG. 1), stored on a computing device memory, such as the computingdevice memory 17 (FIG. 1), may be opened. At processing block 2020, theprojection application may access a first (e.g., high-resolution)pattern, such as the first pattern 19 (FIG. 1), and a second (e.g.,low-resolution) pattern, such as the second pattern 20 (FIG. 1) from thecomputing device memory. The first pattern and the second pattern may bestored in bitmap format.

At processing block 2030, the projection application may perform anexclusive-or (XOR) operation to combine the first pattern and the secondpattern, resulting in a combination pattern bitmap, such as thecombination pattern bitmap 21 (FIG. 1).

At processing block 2040, the projection application may utilize thecombination pattern bitmap and a fabrication device, such as thefabrication device 30 (FIG. 1), to fabricate a combination mask, such asthe combination mask 40 (FIG. 1). At processing block 2050, thecombination mask may be used in conjunction with a projector, such asthe projector 15 (FIG. 1), to generate a returned image, such as thereturned image 22 (FIG. 1). More specifically, the returned image may begenerated by projecting the combination mask onto the target image(i.e., the image for which depth and surface image is to be calculated),and capturing the returned image.

At processing block 2060, the returned image may be passed through alow-pass filter component, such as the low-pass filter component 16(FIG. 1), to generate a second pattern image, such as the second patternimage 23 (FIG. 1). More specifically, the low-pass filter component mayattenuate the high-spatial frequency components in the returned image togenerate an approximation of the image that would have been generatedutilizing the second pattern.

At processing block 2070, the projection application may access and opena preprocessing application, such as the preprocessing application 24(FIG. 1). At processing block 2080, a thresholding function of thepreprocessing application may be utilized to generate a preprocessedreturned image, such as the preprocessed returned image 25 (FIG. 1). Thepreprocessed returned image may be a bitmap version of the returnedimage. At processing block 2090, the thresholding function of thepreprocessing application may be utilized to generate a preprocessedsecond pattern image, such as the preprocessed second pattern image 26(FIG. 1). The preprocessed second pattern image may be a bitmap versionof the second pattern image.

At processing block 2100, the projection application may perform anexclusive-or (XOR) operation to combine the preprocessed returned imageand the preprocessed second pattern image, resulting in a first patternimage, such as the first pattern image 27 (FIG. 1). The generated firstpattern image may be an approximation of the image that would have beengenerated utilizing the first pattern in a projection process. Atprocessing block 2110, the process may terminate.

The sequence and numbering of blocks depicted in FIG. 5 is not intendedto imply an order of operations to the exclusion of other possibilities.Those of skill in the art will appreciate that the foregoing systems andmethods are susceptible of various modifications, variations, andalterations.

For example, in the embodiment described above, the process mayterminate after generating a second pattern image (i.e., processingblock 2060) and a first pattern image (i.e., processing block 2100).However, in other embodiments, an application, such as the projectionapplication, may continue to process the second pattern image and thefirst pattern image to remove image noise.

Embodiments may therefore provide a method of projecting a plurality ofstructured light patterns comprising generating a combination patternbitmap utilizing a first pattern and a second pattern and fabricating acombination mask utilizing the combination pattern bitmap. The methodmay also provide generating a returned image utilizing the combinationmask and generating a second pattern image utilizing the returned image,wherein the second pattern image is an approximation of an image thatwould have been generated utilizing the second pattern, and wherein thegenerating the second pattern image includes passing the returned imagethrough a low-pass filter component.

In one example, the generating the combination pattern bitmap includesperforming an exclusive-or (XOR) operation utilizing the first patternand the second pattern.

In one example, the method may also include generating a preprocessedreturned image and a preprocessed second pattern image and generating afirst pattern image utilizing the preprocessed returned image and thepreprocessed second pattern image, wherein first pattern image is anapproximation of an image that would have been generated utilizing thefirst pattern.

In another example, the preprocessed returned image is generatedutilizing a thresholding function.

In still another example, the preprocessed second pattern image isgenerated utilizing a thresholding function.

In one example, the generating a first pattern image includes performingan exclusive-or (XOR) operation utilizing the preprocessed returnedimage and the preprocessed second pattern image.

Embodiments may also include at least one machine readable mediumcomprising a plurality of instructions that in response to beingexecuted on a computing device, cause the computing device to carry outany of the examples of the aforementioned method, an apparatus forprojection of a plurality of structured light patterns including aninterface and logic configured to perform any of the examples of theaforementioned method and a system for projection of a plurality ofstructured light patterns including an interface, a memory component,and logic configured to perform the method of any of the examples of theaforementioned method.

Still another embodiment may provide a method for projection of aplurality of structured light patterns comprising generating acombination pattern bitmap utilizing a first pattern and a secondpattern, generating a returned image utilizing the combination patternbitmap, and generating a second pattern image utilizing the returnedimage, wherein the second pattern image is an approximation of an imagethat would have been generated utilizing the second pattern;

In one example, the method may also include generating a preprocessedreturned image and a preprocessed second pattern image and generating afirst pattern image utilizing the preprocessed returned image and thepreprocessed second pattern image, wherein first pattern image is anapproximation of an image that would have been generated utilizing thefirst pattern.

In one example, the generating the combination pattern bitmap includesperforming an exclusive-or (XOR) operation utilizing the first patternand the second pattern.

In another example, the method may include fabricating a combinationmask utilizing the combination pattern bitmap.

In still another example, the generating the second pattern imageincludes passing the returned image through a low-pass filter component.

In yet another example, the preprocessed returned image is generatedutilizing a thresholding function.

In one example, the preprocessed second pattern image is generatedutilizing a thresholding function.

In another example, the generating a first pattern image includesperforming an exclusive-or (XOR) operation utilizing the preprocessedreturned image and the preprocessed second pattern image.

Embodiments may also include at least one machine readable mediumcomprising a plurality of instructions that in response to beingexecuted on a computing device, cause the computing device to carry outany of the examples of the aforementioned method, an apparatus forprojection of a plurality of structured light patterns including aninterface and logic configured to perform any of the examples of theaforementioned method and a system for projection of a plurality ofstructured light patterns including an interface, a memory component,and logic configured to perform the method of any of the examples of theaforementioned method.

Embodiments may also provide at least one computer readable storagemedium comprising a set of instructions which, if executed by aprocessor, cause a computing device to generate a combination patternbitmap utilizing a first pattern and a second pattern, generate areturned image utilizing the combination pattern bitmap, and generate asecond pattern image utilizing the returned image, wherein the secondpattern image is an approximation of an image that would have beengenerated utilizing the second pattern.

In one example, the set of instructions may also cause a computer togenerate a preprocessed returned image and a preprocessed second patternimage and generate a first pattern image utilizing the preprocessedreturned image and the preprocessed second pattern image, wherein firstpattern image is an approximation of an image that would have beengenerated utilizing the first pattern.

Embodiments may therefore provide an apparatus comprising an interface,a processing component, and a memory component. The apparatus may alsoinclude a combination pattern module to generate a combination patternbitmap utilizing a first pattern and a second pattern, a returned imagemodule to generate a returned image utilizing the combination patternbitmap, and a second pattern image module to generate a second patternimage utilizing the returned image, wherein the second pattern image isan approximation of an image that would have been generated utilizingthe second pattern. The apparatus may also include a preprocessingmodule to generate a preprocessed returned image and a preprocessedsecond pattern image and a first pattern image module generate a firstpattern image utilizing the preprocessed returned image and thepreprocessed second pattern image, wherein first pattern image is anapproximation of an image that would have been generated utilizing thefirst pattern.

Embodiments may include a system comprising a fabrication component anda computing device. The computing device may include an interface, aprocessing component, and a memory component. The computing device mayalso include a combination pattern module to generate a combinationpattern bitmap utilizing a first pattern and a second pattern, areturned image module to generate a returned image utilizing thecombination pattern bitmap, and a second pattern image module togenerate a second pattern image utilizing the returned image, whereinthe second pattern image is an approximation of an image that would havebeen generated utilizing the second pattern. In addition, the computingdevice may also include a preprocessing module to generate apreprocessed returned image and a preprocessed second pattern image anda first pattern image module generate a first pattern image utilizingthe preprocessed returned image and the preprocessed second patternimage, wherein first pattern image is an approximation of an image thatwould have been generated utilizing the first pattern.

Various embodiments may be implemented using hardware elements, softwareelements, or a combination of both. Examples of hardware elements mayinclude processors, microprocessors, circuits, circuit elements (e.g.,transistors, resistors, capacitors, inductors, and so forth), integratedcircuits, application specific integrated circuits (ASIC), programmablelogic devices (PLD), digital signal processors (DSP), field programmablegate array (FPGA), logic gates, registers, semiconductor device, chips,microchips, chip sets, and so forth. Examples of software may includesoftware components, programs, applications, computer programs,application programs, system programs, machine programs, operatingsystem software, middleware, firmware, software modules, routines,subroutines, functions, methods, procedures, software interfaces,application program interfaces (API), instruction sets, computing code,computer code, code segments, computer code segments, words, values,symbols, or any combination thereof. Determining whether an embodimentis implemented using hardware elements and/or software elements may varyin accordance with any number of factors, such as desired computationalrate, power levels, heat tolerances, processing cycle budget, input datarates, output data rates, memory resources, data bus speeds and otherdesign or performance constraints.

One or more aspects of at least one embodiment may be implemented byrepresentative instructions stored on a machine-readable medium whichrepresents various logic within the processor, which when read by amachine causes the machine to fabricate logic to perform the techniquesdescribed herein. Such representations, known as “IP cores” may bestored on a tangible, machine readable medium and supplied to variouscustomers or manufacturing facilities to load into the fabricationmachines that actually make the logic or processor.

Techniques described herein may therefore provide a feed-forward systemthat ensures both real-time operation of the consumer video pipeline anddynamic updating of the operating pipeline to deliver optimal visualperceptual quality and viewing experience. In particular, a discretecontrol system for the video pipeline can dynamically adapt operatingpoints in order to optimize a global configuration of interactivecomponent modules that are related to video perceptual quality. In aseries configuration, the perceptual quality analysis module may beplaced before the video processing pipeline and parameters determinedfor the post-processing pipeline may be used for the same frame. In thecase of distributed computation of the quality analysis block or whenperceptual quality analysis needs to be performed at intermediate pointsin the pipeline, the parameters determined using a given frame may beapplied on the next frame to ensure real-time operation. Distributedcomputation is sometimes beneficial in reducing complexity as certainelements for perceptual quality computation may already be computed inthe post-processing pipeline and can be re-used. Illustrated approachesmay also be compatible with closed-loop control where the perceptualquality analysis is re-used at the output of the video processingpipeline to estimate output quality, which is also used by the controlmechanism.

Embodiments of the present invention are applicable for use with alltypes of semiconductor integrated circuit (“IC”) chips. Examples ofthese IC chips include but are not limited to processors, controllers,chipset components, programmable logic arrays (PLAs), memory chips,network chips, and the like. In addition, in some of the drawings,signal conductor lines are represented with lines. Some may bedifferent, to indicate more constituent signal paths, have a numberlabel, to indicate a number of constituent signal paths, and/or havearrows at one or more ends, to indicate primary information flowdirection. This, however, should not be construed in a limiting manner.Rather, such added detail may be used in connection with one or moreexemplary embodiments to facilitate easier understanding of a circuit.Any represented signal lines, whether or not having additionalinformation, may actually comprise one or more signals that may travelin multiple directions and may be implemented with any suitable type ofsignal scheme, e.g., digital or analog lines implemented withdifferential pairs, optical fiber lines, and/or single-ended lines.

Example sizes/models/values/ranges may have been given, althoughembodiments of the present invention are not limited to the same. Asmanufacturing techniques (e.g., photolithography) mature over time, itis expected that devices of smaller size could be manufactured. Inaddition, well known power/ground connections to IC chips and othercomponents may or may not be shown within the figures, for simplicity ofillustration and discussion, and so as not to obscure certain aspects ofthe embodiments of the invention. Further, arrangements may be shown inblock diagram form in order to avoid obscuring embodiments of theinvention, and also in view of the fact that specifics with respect toimplementation of such block diagram arrangements are highly dependentupon the platform within which the embodiment is to be implemented,i.e., such specifics should be well within purview of one skilled in theart. Where specific details (e.g., circuits) are set forth in order todescribe example embodiments of the invention, it should be apparent toone skilled in the art that embodiments of the invention can bepracticed without, or with variation of, these specific details. Thedescription is thus to be regarded as illustrative instead of limiting.

Some embodiments may be implemented, for example, using a machine ortangible computer-readable medium or article which may store aninstruction or a set of instructions that, if executed by a machine, maycause the machine to perform a method and/or operations in accordancewith the embodiments. Such a machine may include, for example, anysuitable processing platform, computing platform, computing device,processing device, computing system, processing system, computer,processor, or the like, and may be implemented using any suitablecombination of hardware and/or software. The machine-readable medium orarticle may include, for example, any suitable type of memory unit,memory device, memory article, memory medium, storage device, storagearticle, storage medium and/or storage unit, for example, memory,removable or non-removable media, erasable or non-erasable media,writeable or re-writeable media, digital or analog media, hard disk,floppy disk, Compact Disk Read Only Memory (CD-ROM), Compact DiskRecordable (CD-R), Compact Disk Rewriteable (CD-RW), optical disk,magnetic media, magneto-optical media, removable memory cards or disks,various types of Digital Versatile Disk (DVD), a tape, a cassette, orthe like. The instructions may include any suitable type of code, suchas source code, compiled code, interpreted code, executable code, staticcode, dynamic code, encrypted code, and the like, implemented using anysuitable high-level, low-level, object-oriented, visual, compiled and/orinterpreted programming language.

Unless specifically stated otherwise, it may be appreciated that termssuch as “processing,” “computing,” “calculating,” “determining,” or thelike, refer to the action and/or processes of a computer or computingsystem, or similar electronic computing device, that manipulates and/ortransforms data represented as physical quantities (e.g., electronic)within the computing system's registers and/or memories into other datasimilarly represented as physical quantities within the computingsystem's memories, registers or other such information storage,transmission or display devices. The embodiments are not limited in thiscontext.

The term “coupled” may be used herein to refer to any type ofrelationship, direct or indirect, between the components in question,and may apply to electrical, mechanical, fluid, optical,electromagnetic, electromechanical or other connections. In addition,the terms “first”, “second”, etc. may be used herein only to facilitatediscussion, and carry no particular temporal or chronologicalsignificance unless otherwise indicated.

Those skilled in the art will appreciate from the foregoing descriptionthat the broad techniques of the embodiments of the present inventioncan be implemented in a variety of forms. Therefore, while theembodiments of this invention have been described in connection withparticular examples thereof, the true scope of the embodiments of theinvention should not be so limited since other modifications will becomeapparent to the skilled practitioner upon a study of the drawings,specification, and following claims.

We claim:
 1. A method of projecting a plurality of structured lightpatterns, comprising: projecting a combination pattern bitmap generatedby a first pattern and a second pattern; creating a combination maskutilizing the combination pattern bitmap; capturing, with a sensor, areturned image utilizing the combination pattern bitmap; and generatinga second pattern image utilizing the returned image, by applying alow-pass filter selected from a digital filter, an electronic circuit,and an analog filter to attenuate high spatial frequency components inthe returned image; wherein the returned image is generated byprojecting the combination mask onto a target image.
 2. The method ofclaim 1, wherein the combination pattern bitmap is generated byperforming an exclusive-or (XOR) operation utilizing the first patternand the second pattern.
 3. An apparatus, comprising: a projector toproject a combination pattern bitmap generated by a first pattern and asecond pattern; a mask generator to create a combination mask utilizingthe combination pattern bitmap; a sensor to capture a returned imageutilizing the combination pattern bitmap; and a second pattern generatorto generate a second pattern image utilizing the returned image byapplying a low-pass filter selected from a digital filter, an electroniccircuit, and an analog filter to attenuate high spatial frequencycomponents in the returned image; wherein the returned image isgenerated by projecting the combination mask onto a target image.
 4. Theapparatus of claim 3, wherein the combination pattern bitmap isgenerated by performing an exclusive-or (XOR) operation utilizing thefirst pattern and the second pattern.
 5. At least one computer readablestorage medium comprising a set of instructions which, if executed by aprocessor, cause a computing device to: project a combination patternbitmap generated by a first pattern and a second pattern; create acombination mask utilizing the combination pattern bitmap; capture areturned image utilizing the combination pattern bitmap; and generate asecond pattern image utilizing the returned image by applying a low-passfilter selected from a digital filter, an electronic circuit, and ananalog filter to attenuate high spatial frequency components in thereturned image; wherein the returned image is generated by projectingthe combination mask onto a target image.
 6. The at least one computerreadable storage medium of claim 5, wherein the combination patternbitmap is generated by performing an exclusive-or (XOR) operationutilizing the first pattern and the second pattern.